Hey @FBruzzesi, as I've mentioned 8347839 times, I'm very keen to see this in narwhals! 😄

Would you be okay if we let this one marinate for a bit and maybe target the release after the next? 🙏

Hey @FBruzzesi,as I've mentioned 8347839 times, I'm very keen to see this in narwhals! 😄

🎉

Would you be okay if we let this one marinate for a bit and maybe target the release after the next? 🙏

Yeah I was not expecting this to be finalized by tomorrow. It has some sharp edges that need some love and work

@MarcoGorelli any "trick" to avoid the custom check in precommit?

• For compliant namespaces the import of IntegerType is in the if TYPE_CHECKING block
• For function.py, Int64 is used as default value (and then overwritten by the v*.dtypes.Int64 dtype). I guess one way of doing it is to let it be None and set it inside the function given the version. Yet It's a bit less explicit to a user.

I thought about emulating that utils/import_check.py does, but I would need to figure out what's happening there from scratch 😂

any "trick" to avoid the custom check in precommit?

You could add this to nw.typing, and then use that everywhere instead?

IntegerDType: TypeAlias = "dtypes.IntegerType | type[dtypes.IntegerType]"

this is probably fine, i'd just like to take a look before merging please, thanks 🙏

@MarcoGorelli pinging again as (#2895 (comment)) was a month ago 🫣

@FBruzzesi I still have hope that eventually we'll land this 🙏

I have an idea of how we could reuse int_range to implement date_range for pyarrow.

Idea

As an example, if we convert the output of pl.date_range to pyarrow, we can see the pa.DataType as:

import polars as pl
import pyarrow as pa

start, end = dt.date(2000, 1, 1), dt.date(2000, 1, 5)
expr = pl.date_range(start, end).alias("date")
table = pl.select(expr).to_arrow()
table.column("date").type

See date32

DataType(date32[day])

Since that is just represented as a 32-bit integer, we can do things like:

dates = table.column("date")
dates

<pyarrow.lib.ChunkedArray object at 0x00000296A456BDC0>
[
  [
    2000-01-01,
    2000-01-02,
    2000-01-03,
    2000-01-04,
    2000-01-05
  ]
]

dates.cast(pa.int32())

<pyarrow.lib.ChunkedArray object at 0x00000296A456B8E0>
[
  [
    10957,
    10958,
    10959,
    10960,
    10961
  ]
]

dates.cast(pa.int32()).cast(pa.date32())

<pyarrow.lib.ChunkedArray object at 0x00000296A456BAC0>
[
  [
    2000-01-01,
    2000-01-02,
    2000-01-03,
    2000-01-04,
    2000-01-05
  ]
]

In practice

import datetime as dt

import pyarrow as pa


def date_range(
    start: dt.date,
    end: dt.date,
    interval: int,  # (* assuming the `Interval` part is solved)
    *,
    closed: ClosedInterval = "both",
) -> pa.Date32Array:
    start_i = pa.scalar(start).cast(pa.int32()).as_py()
    end_i = pa.scalar(end).cast(pa.int32()).as_py()

    # call `int_range` here for the compatibility branch
    arr = pa.arange(start_i, end_i + 1, interval)
    if closed != "both":
        if closed == "left":
            arr = arr.slice(length=len(arr) - 1)
        elif closed == "none":
            arr = arr.slice(1, len(arr) - 1)
        else:
            arr = arr.slice(1)

    # the first cast would happen in `int_range(dtype=...)`
    return arr.cast(pa.int32()).cast(pa.date32())


start, end = dt.date(2000, 1, 1), dt.date(2000, 2, 1)

date_range(start, end, interval=7, closed="none")
<pyarrow.lib.Date32Array object at 0x00000296A74176A0>
[
  2000-01-08,
  2000-01-15,
  2000-01-22,
  2000-01-29
]

What's the catch?

We'd need to adapt Interval to support a different kind of parse.

It would be restricted to only d, w, mo, q, y - but even within that - maybe more flexible given that we know +1d is equivalent to +1 🤔?